Time domain circuits for filtering signals expressible as solutions to linear homogeneous differential equations with constant coefficients



IBLE

Apnl 29, 1958 ULE TIME DOMAIN CIRCUITS FOR FILTERINC SIGNALS EXPRESS ASSOLUTIONS TO LINEAR HOMOGENEOUS DIFFERENTIAL EQUATIONS WITH CONSTANTCOEFFICIENTS 3 Sheets-Sheet 1 Filed Sept. 26, 1955 M4664 WM a/ecem rpvs/r/a/v 56 155 1' a Z26) (s ga= gflg 5/1: 2

'f/F a, #a 1 4 4 INVENTOR.

fie. 2.

April 29, 1958 A ULE 2,832,937

TIME DOMAIN CIRCUITS FOR FILTERING SIGNALS EXPRESSIBLE AS SOLUTIONS T0LINEAR HOMOGENEOUS DIFFERENTIAL EQUATIONS WITH CONSTANT COEFFICIENTSFiled Sept. 26, 1955 3 Sheets-Sheet 2 IN V EN TOR.

April 29, 1958 Filed Sept. 26, 1955 A. ULE TIME DOMAIN CIRCUITS FORFILTERING SIGNALS EXPRESS AS SOLUTIONS TO LINEAR HOMOGENEOUSDIFFERENTIAL EQUATIONS WITH CONSTANT COEFFICIENTS 2,832,937 IBLE 5Sheets-Sheet 3 INVENTOR.

T D CIRCUITS FQR FTLTERING SEG- NALS EXPRESSIBLE AS SGLUTIONS T0 LHNEARHOMOGENEOUS DIFFERENTHAL EQUATIONS WITH CONSTANT CUEFFECENTS Louis A.The, Alhambra, Calif., assignor to Giifiiian Bros. Inc., Los Angeles,Calif., a corporation of California Application September-26, 1955,Serial No. 536,612

8 Claims. (ill. 333-749) This invention relates to time domain filteringcircuits and, more particularly, to a class of linear filter circuitsadapted for smoothing, predicting, and other operations in accordancewith a weighted least squares formula. In further particular, theinvention contemplates circuits which may be designed to pass a linearcombination of arbitrary time varying signals which may be specified assolutions to linear homogeneous differential equations with constantcoefiicients.

Other approaches have been made towards providing filters adapted forvarious prediction and smoothing problems. In particular, reference ismade to an article in the Journal of Applied Physics, July 1950, byZadeh and Ragazzini entitled An extension of Wieners theory ofprediction. In this article a solution is obtained for a different typeof problem than that approached herein in that the signals dealt withtherein are specified only in terms of a polynomial series.

The general approach of Zadeh and Ragazzini differs substantially fromthat considered herein since their approach requires that a time variantsignal such as a sinusoid must be treated as an infinite polynomialseries. It will be shown, on the other hand, that such signals may betreated directly in accordance with the present invention and furtherthat a linear combination of a plurality of signals forming solutions tolinear homogeneous differential equations may be filteredsimultaneously.

Another practical limitation in the prediction approach of Zadeh andRagazzini is that the particular solution provided requires a memorydevice which may be a delay line, for its synthesis, and does notachieve prediction and smoothing by means of lumped filter elementsonly. (A lumped filter element is either a coil, a capacitor or aresistor.) This means that additional equipment such as magnetic tape orultrasonic delay lines, is required for the memory device andconsequently the synthesis may not be achieved through simple componentssuch as resistors, inductors, and capacitors, as in embodiments of thepresent invention.

The present invention obviates the above and other disadvantagesinherent in previous approaches in providing filters for operating upontime varying signals which form solutions to linear homogeneousdifferential equations with constant coeflicients. The technique of theinvention thus allows the direct synthesis of circuits which may operateupon various linear combinations of one or more time varying signalssuch as sinusoids, exponentials, step functions, and time varyingsignals of various powers.

According to the theory of the invention, the filters are arranged tominimize the integral:

The functions ag U) are any n linearly independent solutions to aparticular nth order linear homogeneous differential equation withconstant coefiicients. It is pre 2,32,37 Patented Apr. 29, 3.958

sumed further that the input signal M(t) could be exactly represented inthis manner if it were not contaminated by error. In the event of noerror, of course, the minimum value of the integral I is zero. Thesignal function H(t) is an error weighting function which serves twopurposes: (1) It permits us to emphasize the contribution of the dataM(t) to the output circuit of the filter at these times when these dataare known to be most reliable. (2) It permits us to consider an infiniteinterval and consequently by suitable choice of H(t) to real ize thefilter as finite lumped constant apparatus.

Since the quantity 1 being minimized is the weighted squared error, thedesired filter is of the least squares type. The general form of thisfilter may be represented as the Laplace transform of the weightingfunction W(x, t):

where W(x, t) will be shown to be defined by the function:

the parameter x being the prediction time of the filter in seconds. Forzero prediction time the parameter 1: is set to zero although it may bea positive or negative numher. In this expression all terms are definedas above. The matrix of the coefficients cjk is the inverse of thematrix of the coefiicients b defined as follows:

it will be shown in the detailed description which follows that animportant corollary of this theory is that for the particular choice of,H (0:5 where a is a positive real constant, the resulting filters arealways realizable as a driving point impedance to a ladder network oflossy coils and lossy condensers. This will be shown to have two usefulpractical implications:

(1) The ladder may always be synthesized by a continued fractionexpansion of the Laplace transform W(0, t). This synthesis procedureappears to be easier than any other presently known.

(2) Since the particular inductors and condensers specified by theresulting impedance functions are all lossy, the circuit may readily besynthesized with practical components.

in general terms then the invention provides a class of filters foroperating in smoothing and predicting upon an input signal M(t) toderive therefrom the expected output signals which may be referred toherein as S(t), defined as a summation:

The technique of the invention thus provides a least squares approach tothe problem which is weighted through the function H(t) so as to resultin structure which may be synthesized through simple means.

Furthermore, as is more clearly shown below, the prediction parameter xallows a filter of simple passive elements to perform a predictionfunction wherein these elements inherently provide a memory. Thuseflfectively a capacitor is utilized in accordance with the invention tostore information whereas the above-mentioned prediction devices requiremore complicated devices.

Accordingly, it is an object of the present invention to provide filtercircuits adapted for smoothing, predicting, and other operations inaccordance with a weighted least squares formula.

Another object is to provide circuits which may pass one or more timevarying signals which may be specified as solutions to linearhomogeneous differential equations with constant coefiicients.

A further object is to provide filters which are readily realizablethrough practical components and yet are adapted to pass a plurality oflinearly combined time varying signals which may be specified assolutions to linear homogeneous difierential equations with constantcoeificients.

Yet another object is to provide filter circuits capable of predictionbut which do not require delay line or other memory devices but ratherachieve prediction through the inherent memory characteristic of filtercomponents, such as capacitors and thelike.

Still a further object of the invention is to provide a class of filterswhich may be directly synthesized to operate upon a finite number oflinearly combined time varying signals.

A more specific object of the invention is to provide a weighted leastsquares type of filter circuit adapted in a combination of such timevarying signals as sinusoids, exponentials, step functions, and powerfunctions of time, all of which may be solutions to linear homogeneousdifferential equations having constant coefficients.

The novel features which are believed to be characteristic of theinvention, together with further objects, will be better understood fromthe following description considered in connection with the accompanyingdrawings included for illustration only and not as a definition of thelimits of the invention.

Fig. 1a illustrates a typical time domain function which may be filteredin accordance with the present invention;

Figs. 1b and 1c illustrate forms of networks for passing the signalillustrated in Fig. 1a;

Figs. 2 and 3 illustrate synthesesof filters for passing polynominalseries functions of the second and third degrees, respectively;

Fig. 4 illustrates a filter synthesis providing a satisfactoryreproduction of a square wave;

Fig. 5 illustrates ta filter synthesis for the linear combination of aconstant level and exponential signal; and

Fig. 6 illustrates a filter synthesis for passing the linear combinationof three signals;

(a) e (b) e sin 1000i (0) ecos 1000i The discussion which follows willbe separated into four principal parts all of which are also containedin an article by Louis A. Ule entitled Weighted least-squares smoothingfilters, IRE Transactions on Circuit Theory, vol. CIT-2, No. 2, June1955. In part I the technique of least squares curved fitting will begeneralized and modified to introduce an error weighting function. Thediscussion in this part then will not be particularly directed to thefilter problem but rather will develop basic mathematical theory.Following part I the basic theory will be extended to linear filters inpart II. After this the basic weighting function W(t) of the filter willbe derived in part III, and finally the filter design procedure will bediscussed in part IV to indicate the technique of network synthesiswhich is employed to provide actual circuits defined in accordance withthe theory of the invention. This part will include several specificexamples of circuits shown in the various figures of the drawings.

I. Weighted least squared error in. curve fitting A curve can always befitted exacly to a set of data but such a curve is not always the bestrepresentation of the variable of which the data are measurementsn Ifthe variable, for example, must obey some physical law, it may well bethe solution to an nth order differential equation and hence known tohave the form:

where by LC We mean linear combination of and where f (x), f (x), f,,(x)are known functions of the variable x. A set of measurements of F(x),because of unavoidable errors of measurement, will not he representablein the form of Equation 1. A new function S(x) may be found which hasthe form (1) and differs at least in some Way from the given data. S(x)Will then represent a best conclusion drawn from 'two pieces ofevidence: foreknowledge of F(x) and measurements on it. If themeasurements be called M(x) and S(x) is their smoothed value, theweighted squared error for any x will be:

where H(x) is an arbitrarily chosen error weighting function. Thecumulative weighted squared error we define as =f (2) It is thisquantity that is to be minimized.

The introduction of the error weighting function H(x) adds nocomplications and has several advantages. It can be so specified as topermit an infinite interval to be considered and yet have E be finite;'also, H(x) may be made large inthose intervals where the data M(x) aremost significant or where the error of measurement is known to be least.Data M(x) then, instead of being weighed uniformly may be weighedaccording to their validity.

If S(x) is written as 7lx =Z are) and the following is performed 6E VM-0, -1,2, ,7],

for the minimum weighted squared error {a set of n linear simultaneousequations for the coeflicients a is obtained. These are:

Equation 5 provides a series of equations which'may be written asfollows:

The inverse of the symmetrical matrix (b will exist except for someparticular choice of H(x) for the functions f (x). The numbers b are allassumed to be finite. That solutions exist at all is easy to show. If,for example, the f (x) are orthogonal functions with the weight functionH(x) over the interval x 0, the matrix (b will be a diagonal matrixwhose inverse is obtained Q4! by inspection. Designating the inverse ofthe matrix (bij) by (c then, the values of the coeficients are ll.Extension 0 linear filters For a linear filter to perform continuouslythe curve fitting operation outlined above, it would at any instant oftime have taken the data M(x) as its input and produced at its outputthe value S(x). Hence the variable x would always be measured from thepresent. A clock reading time in our time would stand still. Inaddition, the filter cannot be required to weigh future information: ifit is to predict, it must do so on the basis of past information.Finally, for the problem at hand, it must be understood, considering thefilter will be specified by the class of functions f (x) which it is tosmooth. if the filter is not going to have to continually adapt itsinternal structure to fit the signal, these functions f (x) must notcontinually change into new functions. But the input to a filter is inreal time and the input function f, (t+x) at some time other than thepresent (i. e., x=0), may not be a function for which the filter wasdesigned.

For the function f,-(t+x) to belong to the set of functions defined by(1) for all x, we must be able to express it as a linear combination off (t), f (t), f (t). That is, there must exist functions p (x) suchthat:

For x=0, the right hand member of (11) should reduce to f -(t). Assumingthe p (x) to be continuous functions of x and expanding them about zeroin a Taylor series this requires for x small enough that:

P (x)=xq for j=k P bc) l +xq for j=k The numbers g are just C011".?;l1$.Substituting (12) into (11) and letting x approach zero we obtain in thelimit:

The functions which can be smoothed by a particular linear filter mustthen be set of solutions to a particular nth order linear homogeneousdifferential equation with constant coetiicients. These functions areexponentials, sinusoids and polynominals as well as possible sums ofproducts of such functions. We shall call them E func tions rather thanpolynominals because the general solution to a linear homogeneousdifferential equation with constant coefficients is a sum ofexponentials with polynominal terms found in the solution only in thecase of multiple roots. A set of nE functions comprising all thesolutions of nth order linear homogeneous diiferential equation withconstant coefiicients will be called a complete set. Exact filters canbe designed only for complete sets of E functions.

Hi. Weighting function of the filter If Equation 6 is obtained where anew variable of integration y is chosen to avoid confusion. S(x) canhere be considered as the output at the present time [:0 of a filterWhose input is M(y). If M(y) is expressible as a function of the form(3) and if the filter is a least squared error filter, then S(x) shouldexactly reproduce M(y). F or x positive, the filter should exactlypredict M(y) by x seconds, and if x is negative 5(x) should lag M(y) byx seconds. Hence x in (14) is the prediction time of the filter.Therefore (14) may ibe interpreted as a convolution integral giving theoutput 72G of a filter with prediction time x. That is,

where W(x, t) is the weighting function of a filter which predicts by xseconds expressed in conventional form, M is the input and S is theoutput. Identifying (14) and (15 the weighting function of the filter isobtained as the expression:

which can be represented as the product of three matrices as follows:

for 220 and,

Ve (x, t)=0, for t O The weighting function of the smoothing filter isthus obtained which predicts or delays by any desired amount signals forwhich it was designed. Filters which perform other linear operationssuch as obtaining smooth derivatives or integrals are obtained withequal facility. In addition a single filter can be made to performsmoothly any linear combination of these operations on an input signal.

1V Filter design procedure in view of the foregoing a weighted leastsquared error filter can be designed in the following way. Somethingmust first be known about the expected signal. If possible the linearhomogeneous differential equation with constant coefficients of lowestorder which the signal satisfies should be found. If this is notpossible the lowest order suitable approximation should be found (i. e.,n as small as possible) where Fromote); 12.0)] (1 and the functions f(t), f (t) are a complete set of E functions. The next step is to choosethe error weighting function H(r). Typical choices for this function Inall cases H0) is zero for t positive-the filter will not he asked toweigh the future. Furthermore an admissible choice of H0) may dependupon the functions ,t -(r), either for the numbers b to be finite or forthe inverse of the b matrix to exist. The rapidity with which H(z)approaches zero as r approaches minus infinity will specify the memoryof the filter, its speed of response, settling time and whether thefilter will ring or be critically damped. The choice of H(t)proportional to e where e is the natural logarithmic base and a is anypositive real constant is preferred because a filter transfer functionis thus obtained from which filters may be synthesized as a drivingpoint impedance to a ladder network of lossy inductors and capacitors.The value of 7 g a may be reduced by a desired amount to decrease thebandwith of the filter or to make the numbers b finite V or to make theinverse of the b matrix exist.

Having chosen H(t) the next step is to compute the elements of the bmatrix using Equation 7:

o bfi=f fi f,- H w 7) The matrix (b is then inverted. Using the invertedmatrix (c at once the weighting function of the desired filter isobtained by (17):

For network synthesis it is convenient to work with the filter transferfunction. This is simply the Laplace transform of W(x, t); namely,

APPLICATION OF THEORY TO SPECIFIC CIRCUITS To begin with a simpleexample it will be assumed that the signal F(t) is known to obey therelationship:

and that therefore:

F(t) =LC [a ,a t]

where a and a may assume any constant values.

If the weighting function H(t) is made to vanish over an interval forfinite t, it will now be shown that a delay line type of impedance isspecified, such as is required in practicing the technique of Zadah andRagazzini mentioned above. For this illustration it will be assumed thatthe function will weigh uniformly just past one second of its input andignore all previous inputs so that:

Utilizing Equation 7 above we may obtain the elements of the b matrix,where the limits defined in Equation 7 are modified in accordance withthe selection of the weighting function H(t), as follows:

- 0 b12=b21=f 1$dZ=-1/2 0 b, =f :c da:=1/3 where a =a =1 is assumed forsimplicity.

The inverse matrix Cjk is then:

and by (17) the expression for W(x, t) is:

W(a:, t)={u(t)u(t1)}[1-t] 4 6 If prediction is not required, x=0 and Thetransfer impedance of this filter may be found then by solving theLaplace integral:

' providing the transfer function:

The presence of the delay term emakes it apparent that construction ofthis filter will require some sort of delay device.

It will now be shown that by utilizing a continuous error weightingfunction of time H(t) in accordance with the invention the samefiltering may be accomplished with simple lump constant. elements. inthis example it will be assumed that H(z) is defined as follows:

' If the prediction time is zero (3c()) the impedance function thenbecomes 2s+ 1 which is readily synthesized with lumped constant elementsas a driving point impedance to a ladder.

The general form of the time domain function which is being consideredhere and the characteristic ladder are shown in Figs. 1a and 1b,respectively. The particular ladder constants may be derived in a simplemanner by ascertaining the admittance of each element as follows:

The admittance is then provided by a capacitor of farad, the admittanceof term of is provided by a resistor of ohms, and the admittance term ofwill be readily recognized as the series connection of an inductor of 8henrys and a resistor of 4 ohms.

The above example is not intended to illustrate a practical circuit formbut rather the general configuration of the ladder network. A networkwith more practical element values may be obtained, either by animpedance level change, or a bandwidth change or both. Alternaassess?tively the network can be synthesized with feedback amplifiersthe onlypractical choice when the bandwidth is narrow.

The manner in which the feedback amplifier may be employed to simulatethe desired impedance func ion is more specifically illustrated in Fig.where the filter again is designed in accordance with the transferfunction:

It will also be noted that Fig. 10 has been designed to illustrate atypical application for the invention wherein aircraft positioninformation is to be filtered out of radar noise and is known to betrajectory information which may be approximated by a polynomial or, or"

The impedance function achieved throu; feedback amplifier All may bedefined in a manner to be equal to the ratio of the input atrni Y overthe feedback admittance Y This then vides the impedance function In asimilar manner Z2 may be defined as follows:

Y9 10- l Z2 (380) The open loop impedance G then for the arrangement ofFig. lcis found to be:

G(0pen loop)= (33d) and the complete unity gain feedback transferfunction Z(s) is then found to be:

Another way of deriving this particular filter would have been to defineF(t) as the sum of Laguerre polynomials L Since these are orthonormalwith respect to the weight function e over the interval positive I, theb matrix is a unit diagonal matrix. The expression W(t) would thensimply be:

It is then possible to obtain the Weighting function for filters ofpolynomials of the nth degree with an error weighting function e in thismanner. This function becomes:

1 n( k( k( The Laplace transform of this weighting function for theprediction time x=0 is then:

Two other syntheses of this transformation function are illustrated inthe drawing by way of example. Fig. 2 shows the synthesis of Z (s) inthe form of a ladder of lossy coils and lossy condensers. This laddermay be derived as above in (33a) by first determining the adiii) t 2132'. square lVtiVt'i.

mittance input term and then treating the remaining term as animpedance. The other example is the synthesis of a filter utilizing afeedback amplifier, resistors and concensers Where the expected signalis a polynomial in the third degree. in this case the transfer impedanceis:

e mechanization is shown in Pig. 3 where the transunctien has beenbroken up into two factors as work.

.iters ma also be designed for periodic signals, such l s wiliillustrated by assuming that a satisfactory reproduction of the squarewave is found in the fundamental and third harmonics. Assuma simpleexample where the frequencies are 1 and 3 'ans, we then have:

F(Y)=LC [e 'fl 6- 2- 1 (39) To get simple numbers we choose H(z)::2()eThe resulting [i aatrix is then by (7):

Upon inversion of this matrix We obtain the following expression for thefilter weighting function:

The synthesis of this function is shown in Fig. 4 as a driving pointimpedance. it will be understood again that the figure merelyillustrates the general form of the circuit and is not intended torepresent a specific arrangement.

As a somewhat more unusual example, consider PU) to be defined by sothat A choice of an exponential error weighting function would here beconvenient, however, for the numbers b to be finite the exponent must begreater than 2t. We therefore choose it Calculating the elements of theb matrix by (7) and inverting the matrix we arrive at the followingexpression for the filter weighting function:

For zero prediction time This general form of synthesis for thisfunction is shown in Fig. 5.

As a final example consider the design of a filter in accordance withthe invention which will pass the following three components withoutsteady state error:

(a) e (b) 8- sin 1000i ecos 1000! Where the filter has a smoothing timeof about /3 second; i. e. a bandwidth of about /2 cycle per second.

The filter is unconditionally stable when used as a feedback network ina closed loop system due to the fact that error weighting function HQ)is again selected as:

where a is a positive constant. This means, as above, that the transferfunction is realized as an input impedance to a passive ladder andtherefore cannot exhibit more than a 90 phase shift.

The requirement of a smoothing timeof about second implies that:

The inversion of this matrix is tedious. The resulting transfer function(with zero prediction) turns out to be to an approximation close enoughfor any practical purpose:

The transfer function may be divided into separate sections as follows:

Input network:

Feedback network:

Output network:

This can be realized using a single operational amplifier as shown inPig. 6.

The overall response has a gain of'three overthat required. This is nodrawback since in the closed loop, this can be compensated for, ifnecessary, by a loss of /3 in the forward section of the loop-in anycase the loop will be stable regardless of gain.

I claim: a

1. A filter for deriving an output signal 'S(t) from an input datasignal M(t) with a least squared error in the output signal, theusefulsignal information in M0) being known to include a function F(t)in the form of a linear combination of a number of signals, n, formingsolutions f (t), f (t) f (t) to a linear homogeneous dilferentialequation with the corresponding constant coeflicients whereby the signalS(t) may be defined as:

where H(t)is an error weighting function; the minimized error integral Eproviding the relationship:

j=1 2,. .,n from which the constants a, are found to be:

so that the output signal S(t) with minimized'en'oria definable as:

and the weighting function for the circuit is:

where $523M ammo) J=1 k=1 is the product of three matrices [f1( c c f(:5)

nl Q where x is the prediction time of the circuit.

2. The invention as defined in claim 1, wherein said function F0) is apolynomial of the nth degree and the weighting function W (x, t) With anerror weighting function e becomes the Laplace transform of which is 3.The invention as. defined in claim 2, wherein said circuit includes anetwork mechanized in accordance with the Laplace transform:

where s is the Laplace transformbperator, and said Laplace function Z(x,s) is realizable as a finite lumped constant apparatus and thecontribution of the input data M(t) may be emphasized at those timeswhen the data are known to be most reliable.

4. The circuit defined in claim 2 wherein n=1 and the predictionvariable x=;

said Laplace transform function being then:

and being operable to filter an expected signal which is a polynomialfunction of time in the third degree.

6. The invention as defined in claim 1, wherein said expected signal P(t) may be defined as follows:

where LC means the linear combination of the terms e r, e r said circuitcomprising a network mechanized in accordance with the Laplacetransform:

l 4 said Laplace transform function for F0 then becoming as) 8s +24s+72s+56 s +8s 34s 72s+ said prediction function at being equal to 0.

7. The invention as defined in claim 2, wherein said expected signal PU)may be defined as follows:

F(t)=LC [1, 'e

where LC stands for the linear combination of 1 and e said circuitcomprising a network mechanized in accordance with the Laplacetransform:

said Laplace transform function for x=0 then becoming 4s+6 s +5s+6 saidprediction function x being equal to 0.

8. The invention as defined in claim 1, wherein the expected signalincludes three components, viz. e e sin 10001, and ecos 1000)., saidcircuit comprising a network mechanized in accordance with the Laplacetransform:

said Laplace transform for said expected signals then being equal to Noreferences cited.

